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I.  INTRODUCTION 


While  the  United  States  achieves  its  national  security  missions  by 
equipping  its  military  forces  with  the  best  weapons  systems  in  the  world, 
questions  continue  about  the  performance  of  the  defense  acquisition 
system.  How  effective  is  it?  How  can  that  effectiveness  be  objectively 
measured?  Can  we  use  those  measures  to  affect  behaviors  with  appropriate 
incentives  or  determine  which  policies  and  procedures  improve  results  and 
which  turn  out  to  be  misguided? 

Answering  these  questions  requires  more  than  opinion.  It  requires  analysis 
of  unbiased  data  to  discover  insights  into  underlying  effects.  These,  in 
turn,  will  inform  better  policy  and  programmatic  decisions. 

— Hon.  Frank  Kendall  (2013) 


A.  BACKGROUND 

Many  efforts  have  been  made  over  the  past  twenty-five  (25)  years  to  increase  the 
effectiveness  of  acquisition  of  goods  and  services  within  the  Department  of 
Defense  (DOD).  Funding  requirements  are  increasing,  yet  program  resources  seem  harder 
to  find.  Capability  gaps  remain  unaddressed,  yet  the  DOD  base  budget  has  generally 
shown  continued  growth  through  2013,  as  shown  in  Figure  1  (Office  of  the  Under 
Secretary  of  Defense  [Comptroller],  2014,  2015). 
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Note:  Funding  is  shown  in  Current  Year  dollars;  FY17  is  planned. 

Figure  1.  Department  of  Defense  Budget,  Fiscal  Years  (FYs)  2001-2017. 
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As  current  capabilities  age  out  of  the  field,  the  DOD  workforce  must  become 
adept  at  doing  “more  without  more”  (Carter,  2011,  p.  1).  To  drive  towards  this  balance, 
attempts  to  reform  the  acquisition  system  have  continued. 

One  of  the  most  recent  of  these  acquisition  reform  efforts  is  the  recent  update  to 
DOD  Instruction  (DoDI)  5000.02,  released  in  January  2015.  The  DoDI  5000.02  update  is 
intended  to  enhance  discussion  and  thoughtfulness  regarding  acquisition  program 
planning  and  management,  and  to  encompass  the  acquisition  reform  efforts  initiated  since 
the  2008  version  of  the  DoDI  5000.02. 

B.  PROJECT  OBJECTIVES 

The  primary  objective  of  this  research  is  to  determine,  based  on  an  analysis  of 
historical  trends,  what  the  expected  effect  of  the  January  2015  DoDI  5000.02  will  be  on 
acquisition  effectiveness  of  the  Joint  Program  Executive  Office  for  Chemical  and 
Biological  Defense  (JPEO-CBD)  portfolio.  Effectiveness  will  be  measured  primarily  in 
terms  of  schedule  from  project  inception  to  delivery  of  a  capability  to  the  warfighter, 
including  possible  changes  with  respect  to  number  of  in-process  reviews  and  decision 
points,  as  well  as  secondary  metrics  including  number  of  acquisition  documents  required. 

A  secondary  objective  of  this  research  is  to  determine  the  extent  to  which  other 
recent  acquisition  reforms  also  generate  both  intended  and  unintended  potential  impacts 
on  overall  acquisition  performance,  specifically  in  the  JPEO-CBD  community. 

C.  PROJECT  ACTIVITIES 

1.  Problem  Identification 

This  project  will  analyze,  based  on  historical  trends  in  acquisition  reform,  the 
extent  to  which  the  January  2015  issuance  of  DoDI  5000.02  will  likely  increase 
acquisition  effectiveness  within  the  JPEO-CBD  community,  and  the  extent  to  which  it 
may  place  additional  burdens  on  the  JPEO-CBD  to  deliver  capability  to  the  warfighter. 
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2. 


Informed  Foundation 


This  project  will  leverage  the  annual  Performance  of  the  Defense  Acquisition 
System  reports  as  the  basis  for  trend  analysis  and  discussion.  The  three  reports  that  will  be 
referenced  are  from  2013,  2014,  and  2015. 

It  is  important  to  note  that  each  of  the  annual  reports  highlights  a  different  focus: 

•  2013:  total  life-cycle  cost  performance 

•  2014:  contract  incentive  techniques  and  their  effects  on  cost,  schedule,  and 
technical  performance 

•  2015:  analysis  of  broader  factors  that  influence  acquisition  outcomes. 

3.  Data 

In  addition  to  the  Performance  of  the  Defense  Acquisition  System:  2013  Annual 
Report ,  this  project  will  use  a  sampling  of  documentation  collected  from  various 
programs  and  projects  within  the  JPEO-CBD  portfolio.  This  data  will  include  schedules 
depicting  acquisition  phase  length,  funding  requirements,  and  other  acquisition 
documents.  This  project  will  also  demonstrate  changes  in  contractual  workload  within 
two  of  the  Chemical  Biological  Defense  (CBD)  contracting  divisions. 

4.  Analysis 

Analysis  methodology  will  include  a  literature  review  of  the  relevant  documents, 
as  well  as  other  research  findings  concerning  the  effectiveness  of  these  various  reform 
measures.  Additionally,  this  project  will  include  a  comparative  analysis  of  items  that  may 
indicate  other  proximate  measures  of  effectiveness,  such  as  project  schedules  and  number 
of  documents  required. 

D.  PROJECT  BENEFIT 

As  acquisition  reform  efforts  have  intensified,  DOD  is  adjusting  to  “doing  more 

without  more.”  The  information  contained  in  this  project  will  help  establish  the 

foundation  for  JPEO-CBD  to  achieve  its  mission  with  fewer  resources.  It  will  broaden 

understanding  of  the  present  and  future  acquisition  environments  through  exploration  and 

analysis  of  the  effectiveness  of  past  and  current  reform  measures.  This  project  will 
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explore  the  current  strategies  and  provide  recommendations  to  ensure  that  JPEO-CBD  is 
able  to  fulfill  warfighter  needs  as  effectively  as  possible. 

E.  ORGANIZATION  OF  RESEARCH 

This  research  is  organized  into  three  (3)  basic  sections:  (1)  background 
information,  (2)  data,  and  (3)  analysis. 

First,  background  information  is  available  to  help  the  reader  understand  some  of 
the  most  recent  reform  efforts.  Additional  background  information  is  provided  to  help  the 
reader  understand  the  CBD  enterprise  organizational  structure,  with  further  definition  of 
elements  of  the  JPEO-CBD  structure.  This  is  important  to  effectively  narrow  the  scope  of 
the  research  to  the  level  of  the  program  management  offices  within  the  JPEO-CBD  and 
the  DoDI  5000. 02’s  expected  effects  therein. 

Following  the  background  information,  the  data  and  analysis  are  grouped  into 
three  (3)  phases:  (1)  Pre-Better  Buying  Power  (BBP)  1.0,  (2)  Between  BBP  1.0  and  2.0, 
and  (3)  BBP  2.0  through  DoDI  5000.02.  These  topics  are  organized  in  such  a  way  that 
trend  analyses  can  be  performed,  where  appropriate,  on  metrics  demonstrating 
effectiveness.  BBP  3.0  is  discussed  only  in  the  background  section  for  completeness. 
However,  data  from  this  phase  and  beyond  is  not  analyzed  nor  discussed  separately  as  the 
guidance  contained  therein  is  both  geared  towards  technical  excellence  rather  than 
acquisition  program  or  process  efficiency  and  too  recent  to  discern  any  noticeable  effects 
as  of  the  writing  of  this  report. 
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II.  BACKGROUND 


In  these  times  of  extreme  budget  pressures  and  uncertainty... improving 
the  performance  of  the  defense  acquisition  system  is  essential  for  the 


DOD. 


— Hon.  Frank  Kendall  (2014) 


A.  RECENT  REFORM  EFFORTS 

Modern  acquisition  reform  intensified  in  the  1990s  with  the  passing  of  the 
Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  in  1990,  the  Federal 
Acquisition  Streamlining  Act  of  1994  (FAS A),  and  the  Federal  Acquisition  Reform  Act 
(FARA)  in  1996.  These  acts  were  intended  to  drive  more  effectiveness  into  the 
Department  of  Defense’s  acquisition  processes.  Following  the  FARA,  DoDI  5000.02  was 
released  in  2003  and  again  in  2008.  More  recent  endeavors  include  the  Weapon  Systems 
Acquisition  Reform  Act  (WSARA)  (2009)  and  the  BBP  guidance  memorandums  of  2010 
and  2012.  The  most  recent  additions  to  guidance  reform  are  the  January  2015  version  of 
DoDI  5000.02  and  the  third  installment  of  BBP,  released  in  2015  and  referred  to  as  BBP 
3.0.  The  timeline  of  these  reform  efforts  is  shown  in  Figure  2,  Evolution  of  Recent 
Acquisition  Reform. 


Figure  2.  Evolution  of  Recent  Acquisition  Reform 
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1.  Defense  Acquisition  Workforce  Improvement  Act 

The  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA),  established  in 
1990,  sought  to  establish  consistent  training  and  professional  certification  standards  for 
the  acquisition  workforce.  It  emphasized  the  need  to  offer  civilians  greater  opportunities 
for  professional  development  and  advancement  (Garcia  et  al.,  1997,  p.  1).  Garcia  et  al. 
also  note  that  “DAWIA  implementation  allowed  DOD  Components  to  move  forward  to 
meet  the  daily  acquisition  challenges  with  a  more  highly  trained  and  better  equipped 
workforce”  (p.  11). 

In  addition  to  growing  the  civilian  workforce  in  terms  of  professional 
development,  DAWIA  also  reduced  the  size  of  the  acquisition  workforce  in  order  to 
increase  efficiency  (Hunter  et  al.,  2015,  p.  6). 

2.  Federal  Acquisition  Streamlining  Act 

Established  in  1994,  the  Federal  Acquisition  Streamlining  Act  (FAS A)  resulted 
from  recognition  that  the  acquisition  process  had  become  too  complex.  Strategically, 
FASA  sought  “to  develop  a  more  equitable  balance  between  government-unique 
requirements  and  the  need  to  lower  the  government’s  cost  of  doing  business”  (Barry, 
1995,  p.  3).  FASA  intended  to  decrease  acquisition  burden  primarily  through  system 
procurement  by  implementing  a  performance-based  approach.  FASA  further  “reduced 
unique  purchasing  requirements,  increased  the  use  of  simplified  acquisition  procedures 
for  low-cost  procurements  through  raising  the  simplified  acquisition  threshold  to 
$100,000,  and  sought  to  obtain  goods  and  services  faster  in  order  to  reduce  the  in-house 
costs  of  doing  business”  (O’Connell,  2012,  para.  4).  Additionally,  FASA  eliminated  or 
modified  many  obsolete  and  redundant  laws  and  directed  the  use  of  commercial 
specifications  instead  of  military  standards  and  specifications  wherever  practical. 

3.  Federal  Acquisition  Reform  Act 

The  Federal  Acquisition  Reform  Act  (FARA)  of  1996,  later  renamed  the  Clinger- 
Cohen  Act,  “provided  [additional]  guidance  to  ensure  a  fair  and  open  competitive  process 
for  contractor  support  . . .  and  allowed  contracting  officers  more  discretion  when  making 
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competitive  range  determinations.  Additionally,  the  FARA  established  the  Chief 
Information  Officer  role”  (Acquisition  Community  Connection,  2015,  para.  1). 

4.  Department  of  Defense  Instruction  5000.02 

The  DoDI  5000.02  issued  in  2003  made  major  changes  to  the  acquisition 
framework  that  had  been  in  place  since  1971.  It  expanded  phases  and  decision  points,  and 
added  over  thirty  (30)  additional  documents  to  the  process.  DoDI  5000.02  was  then 
reissued  in  2008,  primarily  changing  the  early  phases  of  the  acquisition  process.  It  placed 
additional  emphasis  on  systems  engineering,  as  well  as  integrated  test  and  evaluation 
throughout  the  program  (SM&A,  2009). 

The  2008  version  also  instituted  the  following  changes  from  the  2003  version: 

•  Major  efforts  and  decision  points  were  renamed: 

•  Materiel  Development  Decision  (MDD)  replaced  the  Concept 
Decision,  and  “became  required  regardless  of  where  the  program 
enters  the  acquisition  process”  (GAO,  2009,  p.  20) 

•  Pre-Milestone  (MS)  A  became  the  Materiel  Solution  Analysis 
phase 

•  Pre-MS  B  phase  renamed  to  Engineering  and  Manufacturing 
Development 

•  Post-Critical  Design  Review  replaced  the  Design  Readiness 
Review 

•  Requirements  for  analysis  of  materiel  and  non-materiel  solutions  changed; 
and  the  Milestone  Decision  Authority  began  approving  the  materiel 
solution  at  MS  A  or  program  point  of  entry. 

•  Competitive  prototyping  became  mandatory  during  the  Technology 
Development  phase.  (2009,  p.  20). 

5.  Weapon  Systems  Acquisition  Reform  Act 

The  Weapon  Systems  Acquisition  Reform  Act  (WSARA)  of  2009  brought  about 
major  reforms  to  the  acquisition  process.  Additional  director  positions  within  the  Office 
of  the  Secretary  of  Defense  were  created  to  oversee  and  influence  acquisition  processes. 
Some  of  these  positions  include  the  director  of  cost  assessment  and  program  evaluation, 
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Deputy  Assistant  Secretary  of  Defense  (DASD)  for  developmental  test  &  evaluation, 
DASD  for  systems  engineering,  and  director  of  performance  and  root  cause  analysis. 
WSARA  emphasized  more  rigor  in  determining  requirements,  including  trade-off 
considerations,  and  influenced  acquisition  strategies  by  promoting  competition 
throughout  a  system’s  life  cycle.  Furthermore,  Congressional  reporting  requirements 
increased.  Hunter  et  al.  note  that  “In  signing  the  bill,  President  Obama  said,  ‘The  purpose 
of  this  law  will  be  to  limit  cost  overmns  before  they  spiral  out  of  control.  It  will 
strengthen  oversight  and  accountability  by  appointing  officials  who  will  be  charged  with 
closely  monitoring  the  weapons  systems  we’re  purchasing  to  ensure  that  costs  are 
controlled’”  (2015,  p  12). 

6.  Better  Buying  Power  1.0 

The  first  BBP  memorandum,  signed  by  Mr.  Ashton  Carter  in  2010  and  known  as 
BBP  1.0,  emphasized  life-cycle  affordability  and  controlling  program  cost  growth.  BBP 
1.0  provided  specific  guidance  to  achieve  greater  affordability  through  “[incentivizing 
productivity  and  innovation  in  industry,  [promoting]  real  competition,  [improving] 
tradecraft  in  acquisition  services,  and  [reducing]  non-productive  processes  and 
bureaucracy”  (Carter,  2011,  p.  6). 

7.  Better  Buying  Power  2.0 

The  follow-on  BBP  memorandum,  signed  by  Mr.  Frank  Kendall  in  2012  and 
known  as  BBP  2.0,  again  focused  on  life-cycle  affordability.  Containing  stronger 
language  than  BBP  1.0,  BBP  2.0  increased  emphasis  on  elimination  of  unproductive 
processes  and  bureaucracy.  Additionally,  BBP  2.0  added  guidance  to  “improve  the 
professionalism  of  the  total  acquisition  workforce”  (Kendall,  2012,  p.  7). 

8.  Better  Buying  Power  3.0 

The  third  installment  in  the  Better  Buying  Power  series,  known  as  BBP  3.0  and 
issued  in  2015,  emphasizes  technical  excellence  and  innovation.  Initiatives  contained  in 
BBP  3.0  continue  to  drive  towards  affordability,  savings  opportunities,  and  effective 
incentives.  It  provides  new  focus  areas  in  “technical  excellence,  innovation,  and 
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dominant  military  capabilities”  (Kendall,  2015,  p.  2).  According  to  Deputy  Secretary  of 
Defense  Bob  Work  and  Undersecretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  Frank  Kendall  III,  “whereas  BBP  1.0  and  2.0  focused  on  reforming  [the] 
acquisition  processes  and  making  them  more  efficient,  BBP  3.0  is  primarily  about 
providing  dominant  capabilities  to  the  warfighter  to  try  to  maintain  the  technological 
overmatch  that  [the  United  States  has]  always  enjoyed  and,  if  anything,  to  try  to  extend  it 
if  at  all  possible”  (2015,  para.  16). 

9.  DOD  Instruction  (DoDI)  5000.02 

The  2015  reissuance  of  DOD  Instruction  5000.02,  initially  presented  in  an  interim 
version  in  November  2013  and  released  in  its  current  version  in  January  2015,  is  one  of 
the  most  recent  acquisition  reform  efforts.  There  are  four  (4)  overarching  objectives: 

•  “Decrease  emphasis  on  ‘rules’  and  increase  emphasis  on  process  intent 
and  thoughtful  program  planning 

•  Provide  program  structures  and  procedures  tailored  to  the  dominant 
characteristics  of  the  product  being  acquired  and  to  unique  program 
circumstances 

•  Enhance  the  discussion  of  program  management  responsibility  and  key 
supporting  disciplines 

•  Institutionalize  changes  to  statute  and  policy  since  the  last  issuance  of 
DoDI  5000.02.”  (National  Defense  Industrial  Association  [NDIA],  2015, 

p.  2) 

The  2015  DoDI  5000.02  focuses  on  critical  thinking,  program  tailoring,  and 
codifying  the  reform  efforts  contained  in  the  BBP  guidance  documents.  Changes  include, 
but  are  not  limited  to: 

•  “Core  document  tone  to  more  clearly  communicate  management  intent, 
from  compliance  to  thoughtful  planning 

•  Notes  added  to  tables  to  clarify  requirements 

•  Rewritten  and  refocused  acquisition  process  procedures,  and 
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•  Substantial  revisions  to  all  enclosures,  to 

•  Accommodate  [acquisition]  community  concerns 

•  Clarify/streamline  procedures.”  (NDIA,  2015,  p.  3) 

B.  UNDERSTANDING  THE  PERFORMANCE  OF  THE  DEFENSE 

ACQUISITION  SYSTEM  ANNUAL  REPORTS 

1.  Performance  of  the  Defense  Acquisition  System:  2013  Annual  Report 

The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (USD  (AT&L))  published  the  Performance  of  the  Defense  Acquisition  System: 
2013  Annual  Report  in  June  2013.  The  purpose  of  this  report  is  to  “provide  improved 
insights  into  key  aspects  of  total  life-cycle  cost  performance  to  inform  policy  decisions 
about  the  overall  acquisition  system  and  institution”  (Office  of  the  Under  Secretary  of 
Defense,  Acquisition,  Technology  and  Logistics  [OUSD  AT&L],  2013,  p.  2).  The  2013 
Annual  Report  focuses  on  the  development  and  early  production  life-cycle  phases,  as 
data  is  most  readily  available  for  these  phases. 

2.  Performance  of  the  Defense  Acquisition  System:  2014  Annual  Report 

Published  in  June  2014,  the  second  annual  report  both  updates  the  prior  year’s 
analysis  and  newly  focuses  on  contract  incentives  for  development  and  early  production 
contracts.  The  report  “examines  various  incentive  techniques  to  see  how  effective  they 
are  at  driving  cost,  schedule,  and  technical  performance”  (2014,  p.  iii). 

3.  Performance  of  the  Defense  Acquisition  System:  2015  Annual  Report 

USD  (AT&L)  published  the  third  annual  report  in  September  2015.  This  2015 
Annual  Report  continues  to  build  upon  prior  year  analyses.  This  continuity  provides 
additional  confidence  that  changes  seen  in  the  prior  year  reports  are  real.  OUSD  AT&L 
notes  that  “new  analyses  have  been  added  to  begin  examining  further  inputs,  factors  that 
affect  outcomes,  and  trends  in  inferred  measures  of  complexity  and  risk”  (2015,  p.  xiv). 
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4.  Data  Interpretation  of  the  Annual  Reports 

Each  of  the  three  annual  reports  presents  the  results  of  data-driven  analysis; 
however,  “interpretation  of  performance  and  the  implication  for  policies  are  left  largely 
to  the  reader”  (2013,  p.  2).  Many  analyses  in  each  report  are  in  the  initial  stages  and 
indicate  areas  for  future  work.  However,  others  are  beginning  to  provide  a  consistent  data 
set  and  provide  increasing  confidence  in  the  trends  shown  therein. 

The  analyses  presented  in  the  report  consist  of  two  main  types  of  performance 
data:  program-level  data,  and  contract-level  data. 

5.  Scope  of  the  Annual  Reports  as  it  Relates  to  JPEO-CBD 

Each  annual  report  uses  data  from  several  Major  Defense  Acquisition  Programs 
(MDAPs),  which  are  defined  as  programs  that  meet  or  exceed  the  Acquisition  Category 
(ACAT)  I  requirements  as  defined  in  DoDI  5000.02.  Although  the  JPEO-CBD  does  not 
have  any  ACAT  I  programs,  the  trends  can  be  reasonably  applied  to  programs  and 
projects  within  the  JPEO-CBD’ s  portfolio. 

Therefore,  the  intent  of  this  thesis  is  to  analyze  and  interpret  these  results  as  they 
pertain  to  the  JPEO-CBD.  This  information  will  be  presented  in  Chapter  III. 

Following  the  presentation  of  the  report  findings,  I  will  correlate  these  results  to 
the  changes  in  DoDI  5000.02.  Are  the  changes  to  DoDI  5000.02,  as  presented  in  the 
January  2015  version,  based  on  sound  analyses? 

C.  CBDP  ENTERPRISE  STRUCTURE 

Established  in  1994,  the  CBDP  was  founded  to  prioritize  CBD  efforts  within  the 
DOD  and  to  improve  coordination  of  CBD  doctrine,  requirements,  research, 
development,  and  acquisition  among  DOD  and  the  military  services  (Gebicke,  1996,  p. 
3).  Figure  3  shows  the  current  Enterprise  structure  of  the  CBDP,  through  the  General 
Officer/Senior  Executive  Service  Level.  These  organizations  work  together,  under  the 
guidance  of  the  Army  as  the  Executive  Agent,  to  ensure  that  warfighters  are  properly 
prepared  to  respond  to  medical  and  non-medical  CBD  hazards. 
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Joint  Program 
Executive  Office  for 
Chemical  and 
Biological  Defense 


Figure  3.  CBDP  Enterprise  Organization 


D.  JPEO-CBD  ORGANIZATIONAL  STRUCTURE 

Prior  to  BBP  1.0,  the  JPEO-CBD  was  organized  into  eight  (8)  Joint  Project 
Manager  (JPM)  offices.  Various  offices  within  JPEO-CBD  supported  the  JPMs.  Prior  to 
BBP  2.0,  the  JPEO-CBD  reorganized  into  seven  (7)  JPM  offices  and  supporting 
headquarter  functions,  as  shown  in  Figure  4.  This  organizational  structure  remained  in 
place  until  March  2016. 
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Figure  4.  JPEO-CBD  Organizational  Structure  through  February  2016. 

Figure  5  depicts  the  most  recent  changes  to  the  JPEO-CBD  organizational 
structure.  This  reorganization  occurred  in  March  2016,  and  decreased  the  number  of  JPM 
offices  from  seven  (7)  to  five  (5).  JPM  Guardian,  JPM  Radiological  &  Nuclear  Defense, 
and  JPM  Elimination  were  consolidated  into  one  JPM,  called  JPM  Guardian.  The  other 
four  (4)  JPMs  remain  intact.  The  reasons  for  these  reorganizations  are  not  relevant  to  this 
project,  and  are  provided  to  increase  reader  understanding  of  the  Chemical  Biological 
Defense  Enterprise. 
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Figure  5.  JPEO-CBD  Organizational  Structure  after  March  2016 


These  JPM  offices  can  be  categorized  into  medical  acquisition  and  non-medical 
materiel  acquisition.  This  distinction  is  important,  as  each  has  unique  requirements. 
Medical  acquisition  programs,  managed  and  funded  under  Medical  Countermeasure 
Systems,  have  additional  requirements  through  other  federal  entities  such  as  the  Food  and 
Drug  Administration  that  the  non-medical  programs  and  projects  are  not  required  to 
adhere  to. 
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III.  ACQUISITION  REFORM  DATA  AND  ANALYSIS 


A.  PRE-BBP  1.0 

1.  Contract  Total  Cost  Growth  Trends:  Development  Phase  Data 

The  2013  Annual  Report  analyzed  total  cost  growth  from  1992-2011.  Total  cost 
growth  is  a  function  of  both  work  scope  increase  and  cost-over-target  growth.  The 
contract  total  cost  growth  in  then-year  dollars  is  shown  in  Figure  6. 
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Figure  6.  DOD  Total  Cost  Growth:  Development  Contracts  (1992-201 1). 

Source:  OUSD  AT&L  (2013,  p.  43). 

Although  this  data  is  through  2011,  it  is  included  in  the  Pre-BBP  1.0  section  for 
two  (2)  reasons:  1)  the  vast  majority  of  the  data  represents  the  time  before  BBP  1.0  was 
issued,  and  2)  the  changes  influenced  by  BBP  1.0  would  not  have  had  noticeable  effect 
on  any  trends  only  one  (1)  year  after  issue.  Note  that  the  trend  line  is  a  simple  linear 
regression  through  all  cost  growth  points.  The  2014  and  2015  annual  reports  each 
provided  one  additional  data  point  regarding  the  total  cost  growth  for  development 


15 


contracts.  The  trend  line  remains  statistically  significant,  according  to  the  2015  Annual 
Report,  and  continues  to  trend  downwards. 

2.  Contract  Cost  Growth  Trends:  Development  Phase  Analysis 

The  graph  shown  in  Figure  6  shows  a  slight  downward  trend  in  contract  total  cost 
growth  for  development  phase  contracts.  The  reduction  was  18  percentage  points  over  19 
years.  It  is  important  to  note  that  many  of  the  more  recent  contracts  have  not  yet  been 
completed,  and  are  at  risk  of  ending  with  higher  cost  growth  than  currently  anticipated, 
thus  potentially  altering  the  trend  line. 

The  data  shows  that  total  contract  cost  growth  has  been  trending  favorably  for 
over  twenty  (20)  years.  The  authors  of  the  2013  Annual  Report  also  note  that  no  single 
contract  type  yields  better  cost  control  (2013,  p.  109).  As  with  the  2013  data,  there  are 
significant  differences  in  cost-growth  among  development  contracts  within  both  the  2014 
and  2015  data  sets.  OUSD  AT&L  notes  that  “the  outlier  cases  must  be  prevented  going 
forward”  (2014,  p.  49). 

B.  BETWEEN  BBP  1.0  AND  BBP  2.0 

1.  Performance  Assessments  and  Root  Cause  Analyses  (PARCA)  Data 

The  Office  of  Performance  Assessments  and  Root  Cause  Analyses  (PARCA)  is 
responsible  for  conducting  root  cause  analyses  for  all  “critical”  Nunn-McCurdy  breaches 
and  those  requested  by  the  Secretary  of  Defense.  Per  the  final  report,  Performance  of  the 
Defense  Acquisition  System:  2013  Annual  Report ,  PARCA  conducted  18  such  analyses, 
from  2010  to  2012,  that  are  summarized  in  Table  1. 
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Table  1.  PARCA  Root  Causes  Analyses  (Statutory  and  Discretionary;  2010-2012) 

Source:  OUSD  AT&L  (2013,  p.  34). 


Dominant  Cause 

10  of  18  Poor  management  performance 

•  Poor  requirements  traceability 

•  Ineffective  use  of  contractual  incentives 

•  Poor  Risk  Management 

•  Poor  Situational  Awareness  within  the  chain  of  command 

5  of  18  Baseline  cost  and  schedule  estimates 

•  Invalid  framing  assumptions 

4  of  18  Change  in  Procurement  Quantity 


Infrequent 

1  of  18  Immature  technology,  excessive  manufacturing  or  integration  risk 

2  of  18  Unrealistic  performance  expectations 

1  of  18  Unanticipated  design,  engineering,  manufacturing,  or  technology 

issues 

None  Funding  inadequacy  or  instability 


In  just  over  half  of  the  cases  analyzed,  the  root  cause  can  be  categorized  as  poor 
management  performance  or  effectiveness  (2013). 

The  Performance  of  the  Defense  Acquisition  System:  2015  Annual  Report 
provides  additional  insight  into  the  frequency  of  Nunn-McCurdy  Breaches,  as  shown  in 
Figure  7. 
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NOTE:  Breaches  due  to  quantity  changes  are  based  on  root-cause  analyses  performed  by  the  Office  of 
Performance  Assessments  and  Root  Cause  Analysis  (PARCA)  in  AT81L  Since  PARCA  was  not  established  until  the 
Weapon  System  Acquisition  Reform  Act  (WSARA)  of  2009,  it  is  unknown  whether  quantity  changes  were  a  root 
cause  of  breaches  before  2009.  There  is  a  statistically  significant  downward  trend  in  both  total  critical  breaches 
and  nonquantity-related  critical  breaches  since  2009.  Breaches  are  after  adjusting  for  inflation.  Since  it  usually 
takes  a  few  years  before  a  program  might  breach  again,  we  removed  programs  from  the  portfolio  count  that  have 
breached  recently  to  avoid  the  potential  bias  towards  an  artificially  low  breach  rate.  Also,  relatively  new  programs 
that  have  not  spent  at  least  30  percent  of  their  original  EMD  schedule  are  not  included  because  they  are  unlikely  to 
breach,  and  thus  would  bias  the  results  downward. 


Figure  7.  Program  Cost-Related  Performance:  Nunn-McCurdy  Breaches  (2007- 

2014).  Source:  OUSD  AT&L.  (2015,  p.  xxii). 

2.  PARCA  Analysis 

The  2013  PARCA  data  shows  a  disturbing  trend:  “poor  management 
performance  or  poor  management  effectiveness  is  the  root  cause  of  just  over  half  of  all 
breaches  analyzed.  Problem  areas  include  poor  systems  engineering  to  translate  user 
requirements  into  testable  specifications,  ineffective  use  of  contractual  incentives,  poor 
risk  management,  poor  situational  awareness  across  the  life-cycle,  unrealistic  cost  and 
schedule  estimates,  and  quantity  changes”  (2013,  p.  34). 

However,  the  2015  Report  provides  a  more  positive  outlook.  Although  non¬ 
quantity  changes  are  the  primary  root  cause  of  all  Nunn-McCurdy  breaches,  and  those 
causes  remain  generally  the  same  as  the  2013  report,  the  frequency  of  those  breaches  is 
decreasing  as  shown  in  Figure  7.  The  report  notes  that  there  is  anecdotal  evidence  that 
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programs  are  showing  improved  execution,  which  may  be  tied  to  initiatives  such  as  BBP 
that  continue  to  raise  cost  consciousness. 


C.  POST-2015  DODI  5000.02 

1.  JPEO-CBD  Program  Schedules 

a.  Medical  Acquisition  Portfolio  Programs 

The  JPEO-CBD  medical  acquisition  portfolio  contains  a  number  of  programs. 
One  such  program  is  a  therapeutics  program  called  Bioscavenger-Plasma  (BSCAV-P). 
BSCAV-P  is  a  plasma-derived  prophylactic  regimen  designed  to  protect  the  warfighter 
from  incapacitation  and  death  caused  by  traditional  and  non-traditional  nerve  agents. 

Data  is  available  for  BSCAV-P  beginning  in  FY  2011.  In  December  2010,  the 
BSCAV  program  office  presented  an  In-Process  Review  (IRP)  decision  briefing  to  the 
Milestone  Decision  Authority  (MDA),  which  presents  MS  B  as  occurring  in  1QFY12  and 
MS  C  occurring  in  3QFY17.  The  MS  B  decision  and  resulting  APB  occurred  in  4QFY12, 
more  specifically  September  2012.  The  program  Fife  Cycle  Management  Plan  (FCMP), 
signed  by  the  MDA  in  December  2011,  presents  MS  B  occurring  in  3QFY12  and  MS  C 
in  1QFY18.  According  to  an  updated  APB  from  November  2013,  the  objective  date  for 
MS  C  Fow  Rate  Initial  Production  (FRIP)  is  1QFY19.  These  dates  are  presented  in 
Table  2. 


Table  2.  BSCAV-P  Schedule 


Event 

Schedule  as  of 
2011 

Schedule  as  of 
2012 

Schedule  as  of 
2013 

Delta 

Milestone  B 

1QFY12 

3QFY12 

4QFY12 

+  3  Quarters 

Milestone  C 

3QFY17 

1QFY18 

1QFY19 

+  4  Quarters 

Another  medical  acquisition  program  is  the  Next  Generation  Diagnostic  System 
(NGDS).  The  NGDS  is  a  portable  biological  pathogen  diagnostic  and  identification 
system  that  will  replace  the  currently  fielded  Joint  Biological  Agent  Identification  and 
Diagnostic  System.  Increment  1  is  a  direct  replacement;  Increment  2  will  address 
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objective  threats  using  multiple  capability  sets.  There  are  multiple  increments,  which  are 
combined  for  simplicity. 

The  NGDS  Increment  1  LCMP,  signed  in  June  2012  at  MS  A,  indicates  multiple 
IPRs  on  the  path  to  MS  C,  scheduled  to  occur  in  2QFY15.  As  of  the  March  2016  IPR 
brief,  MS  C  for  Increment  1  is  scheduled  for  1QFY17.  This  brief  also  indicates  that  MS  B 
for  Increment  2  was  scheduled  for  3QFY16.  According  to  updated  information  received 
from  JPEO-CBD,  the  current  schedule  for  MS  B  for  Increment  2  is  3QFY17.  These  dates 
are  presented  in  Table  3. 


Table  3.  NGDS  Schedule 


Event 

Schedule  as  of 
2012 

Schedule  as  of 
2015 

Schedule  as 
of  2016 

Delta 

Milestone  B 
Increment  2 

3QFY16 

3QFY17 

+  4  Quarters 

Milestone  C 
(FRP) 

Increment  1 

2QFY15 

4QFY16 

1QFY17 

+  7  Quarters 

b.  Non-Medical  Acquisition  Portfolio  Programs 

The  Next  Generation  Chemical  Detector  (NGCD)  program  is  contained  within  the 
JPEO-CBD  non-medical  acquisition  portfolio.  The  program  began  in  FY  2011  to 
improve  the  warfighter’s  ability  to  detect  and  identify  non- traditional  agents,  chemical 
warfare  agents,  and  toxic  industrial  chemicals  in  air  and  on  surfaces.  The  NGCD 
schedule  is  shown  in  Table  4. 


Table  4.  Next  Generation  Chemical  Detector  Schedule 


Event 

Schedule  as  of 
2011 

Schedule  as  of 
2015 

Delta 

Milestone  A 

3QFY13 

3QFY13 

No  change 

Preliminary  Design 
Review 

4QFY16 

4QFY16 

No  change 

Milestone  B 

1QFY17 

3QFY17 

+  2  Quarters 
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The  Joint  Biological  Tactical  Detection  System  (JBTDS)  program,  designed  to 
provide  a  tactical  biological  warfare  agent  detection,  collection,  identification,  and 
networking  capability  to  the  Joint  warfighter,  also  began  in  FY11.  The  MS  B  Decision 
Review,  presented  to  the  MDA  in  October  2014,  indicates  that  MS  B  occurred  at  that 
time  with  MS  C  scheduled  in  4QFY17.  The  program  office  presented  an  IPR  decision 
brief  in  March  2016  indicating  that  MS  C  is  delayed  until  4QFY18.  The  JBTDS  schedule 
is  shown  in  Table  5. 


Table  5.  Joint  Biological  Tactical  Detection  System  Schedule 


Event 

Schedule  as  of 

2014 

Schedule  as  of 

2015 

Delta 

Milestone  B 

1QFY15 

1QFY15 

No  change 

Milestone  C 

4QFY17 

4QFY18 

+  4  Quarters 

All  program  data  was  compiled  from  multiple  sources  within  JPEO-CBD,  as  the 
data  was  not  available  from  a  single  database  or  repository. 

2.  Program  Schedule  Analysis 

The  schedules  shown  in  Tables  2  through  5  are  a  small  representative  sample  of 
both  the  medical  and  non-medical  portfolios  within  JPEO-CBD.  All  four  programs 
experienced  at  least  six  (6)  months,  or  two  (2)  quarters,  of  schedule  growth  between  2011 
and  2016. 

The  cause  of  these  schedule  changes  remains  unclear,  however.  Although  no 
program  appears  to  have  gained  efficiency,  which  would  be  reflected  in  a  negative 
schedule  change,  it  is  not  obvious  what  caused  these  schedule  slips.  One  possibility  for 
the  schedule  growth  may  be  additional  requirements  provided  in  the  2015  DoDI  5000.02. 
Other  possible  causes  for  these  schedule  changes  could  be  additional  user  requirements, 
more  stringent  or  new  testing  requirements,  or  less  funding  in  the  intervening  years  to 
complete  the  same  work. 


21 


Additionally,  a  number  of  programs  within  both  the  medical  and  non-medical 
acquisition  portfolios  have  not  had  any  schedule  changes  in  recent  years.  However,  there 
were  no  programs  within  JPEO-CBD  that  experienced  positive  schedule  impacts.  In  other 
words,  no  program  schedules  showed  milestones  occurring  earlier  than  originally 
planned.  This  indicates  that  recent  acquisition  reforms  have  not  caused  program 
schedules  within  the  JPEO-CBD  portfolios  to  increase  in  efficiency  and  provide 
capability  to  the  warfighter  sooner  than  planned. 

3.  Documentation  Requirements  Data 

Table  6  compares  documentation  requirements  for  selected  documents  necessary, 
either  by  statue  or  regulation,  for  AC  AT  III  programs  as  stated  in  the  2008  and  2015 
versions  of  DoDI  5000.02.  Data  focuses  on  ACAT  III  programs,  as  these  are  the  most 
common  ACAT  level  in  JPEO-CBD. 
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Table  6.  Selected  Document  Comparison,  DoDI  5000.02  (2008)  vs.  DoDI  5000.02  (2015) 


Life  Cycle  Event 

DoDI 

5000.02 

Dev 

FRP/ 

Information 

Issue 

CDD 

RFP 

FD 

Requirement 

Date 

MDD 

MSA 

Val 

release 

MSB 

MS  C 

Dec 

Other 

Acquisition  Decision 

2015 

X 

X 

X 

X 

X 

X 

X 

Memo  (ADM) 

2008 

X 

X 

X 

X 

Acquisition  Program 

2015 

X 

X 

X 

X 

X 

Baseline 

X 

X 

X 

Acquisition  Strategy 

2015 

X 

X 

X 

X 

X 

X 

X 

2008 

Affordability 

2015 

X 

X 

X 

X 

X 

X 

Analysis 

X 

X 

Analysis  of 

2015 

X 

X 

X 

X 

Alternatives  (AoA) 

X 

X 

X 

X 

AoA  Study 

Guidance  &  Study 

2015 

X 

Plan 

2008 

X 

Capability 

Development 

2015 

X 

X 

X 

X 

X 

Document  (CDD) 

2008 

X 

Capability 

Production 

2015 

X 

Document 

2008 

X 
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Table  6  (continued) 


Clinger-Cohen  Act 

2015 

X 

X 

X 

X 

Compliance 

2008 

X 

X 

X 

X 

Cooperative 

2015 

X 

X 

X 

Opportunities 

2008 

X 

X 

X 

Core  Logistics 

2015 

X 

X 

X 

Determination 

2008 

X 

X 

Cyber  Security 

X 

X 

X 

X 

X 

Strategy 

2015 

X 

X 

X 

X 

Exit  Criteria 

2008 

X 

X 

X 

X 

Information  Support 

X 

X 

X 

Plan 

X 

X 

IT  &  NSS 

2015 

X 

Interoperability 

Certification 

2008 

X 

Initial  Capabilities 

X 

Document 

X 

X 

X 

IUID 

X 

X 

X 

X 

Implementation  Plan 

X 

X 

X 

Life-Cycle 

X 

X 

X 

X 

X 

X 

Sustainment  Plan 

X 

X 

2015 

X 

X 

LFT&E  Report 

2008 

X 
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Table  6  (continued) 


LRIP  Quantity 

2015 

2008 

X 

X 

X 

2015 

X  X 

X 

Market  Research 

2008 

X 

X 

OTA  Report  of 

X 

X 

OT&E  Results 

X 

X 

X 

2015 

X 

X 

X 

OMS/MP 

2008 

PESHE  and  NEPA/ 

2015 

X 

X 

X 

EO  12114 

Compliance 

Schedule 

2008 

X 

X 

X 

Post  Implementation 

X 

Review 

X 

Program  Protection 

X 

X 

X 

X 

X 

Plan 

X 

X 

X 

Request  for  Proposal 

X 

X 

X 

X 

(REP) 

2015 

X 

X 

X 

X 

X 

Should-Cost  Target 

2008 

SBIR/STTR 

2015 

X 

X 

X 

X 

X 

Program 

Technologies 

2008 
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Table  6  (continued) 


Spectrum 

2015 

X 

X 

X 

X 

Supportability  Risk 

Assessment 

2008 

X 

X 

X 

System  Threat 

2015 

X 

X 

X 

X 

Assessment  Report 

(STAR) 

2008 

X 

X 

Systems  Engineering 

2015 

X 

X 

X 

X 

Plan  (SEP) 

2008 

X 

X 

X 

Technology 

2015 

X 

X 

X 

Readiness 

Assessment 

2008 

X 

X 

Test  &  Evaluation 

X 

X 

X 

X 

X 

X 

X 

X 
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Table  6  presents  thirty-four  (34)  selected  documents  and  eight  (8)  potential 
decision  points  that  may  require  those  documents.  In  each  DoDI  5000.02  version,  2008 
versus  2015,  there  are  272  occurrences  where  documents  could  be  required.  Of  the 
potential  occurrences,  seventy  (70)  documents  or  document  updates  were  required  for 
ACAT  III  programs  in  the  2008  version  of  DoDI  5000.02;  one-hundred-twenty-two  (122) 
documents  or  document  updates  were  required  in  the  2015  version 

4.  Documentation  Requirements  Analysis 

Over  all  of  the  eight  decision  points,  documentation  requirements  increased  by 
almost  20%  between  the  2008  and  2015  versions  of  DoDI  5000.02.  Some  documents, 
such  as  the  Acquisition  Decision  Memorandum  (ADM)  and  Capability  Development 
Document  (CDD)  updates  are  required  more  often  now  than  in  the  past.  There  are  also 
some  new  documents  required,  such  as  the  Cyber  Security  Strategy  and  Should-Cost 
Targets. 

The  expected  effect  of  the  increased  documentation  requirements  is  not  an  overall 
increase  in  acquisition  effectiveness  or  efficiency.  More  program  staff  will  be  required  to 
complete  the  documents,  which  will  drive  up  the  cost  of  a  program.  Alternatively,  no 
additional  staff  will  be  hired  and  the  work  will  be  completed  by  current  team  members. 
Because  the  new  documents  are  additional  duties,  the  program  schedule  may  slip  as  staff 
will  have  to  prioritize  this  additional  work.  In  the  worst  case,  important  management  or 
oversight  may  be  neglected  due  to  shifting  priorities. 

5.  CBD  Contractual  Workload  Data 

Throughout  the  CBDP  enterprise,  there  are  a  number  of  contracting  commands 
that  provide  services.  The  Army  Contracting  Command-Aberdeen  Proving  Ground 
(ACC-APG)  is  the  parent  contracting  organization  for  ten  (10)  contracting  centers  that 
provides  a  number  of  contracting-related  services  to  the  Army.  Of  these  ten  (10) 
contracting  centers  that  service  the  Army,  five  (5)  provide  full-spectrum  contracting 
services  to  JPEO-CBD.  Figure  8  shows  the  ACC-APG  and  subordinate  command 
organizational  structure  that  the  JPEO-CBD  uses.  Additional  centers  not  shown  in 
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Figure  8  include  ACC-Adelphi,  ACC-Denver,  ACC-Ft.  Detrick,  ACC-Orlando,  and 
ACC-Tobyhanna. 


Figure  8.  ACC-APG  Organizational  Structure  Supporting  JPEO-CBD 


ACC-APG  provided  comprehensive  data  for  FY12-FY15  for  this  project.  As  the 
organizational  structure,  and  therefore  record  keeping,  changed  in  FY12,  comprehensive 
data  is  not  available  for  FY11  and  prior.  Table  7  provides  a  summary  of  the  total  number 
of  actions  performed  by  ACC-APG;  the  total  number  of  those  actions  that  can  be 
attributed  to  JPEO-CBD;  the  total  funds  obligated  by  APG-ACC;  the  funds  obligated  that 
can  be  attributed  to  JPEO-CBD;  and  the  percentage  of  work,  measured  in  both  number  of 
actions  and  funds  obligated,  that  are  attributed  to  JPEO-CBD. 


Table  7.  ACC-APG  and  JPEO-CBD  Contract  Actions  and  Funding  Obligations, 

FY12-FY15 


Fiscal 

Year 

Total 

ACC 

Actions 

JPEO 

Actions 

JPEO 

Actions 

(%) 

Total 

Obligations  ($) 

JPEO 

Obligations 

($) 

JPEO 

Obligations 

(%) 

FY12 

37,153 

1,083 

2.91% 

$15,106,694,543 

$740,867,335 

4.90% 

FY13 

31,248 

1,221 

3.91% 

$12,680,685,618 

$590,214,092 

4.65% 

FY14 

30,356 

1,494 

4.92% 

$10,897,392,830 

$430,112,125 

3.95% 

FY15 

30,533 

1,637 

5.36% 

$9,975,741,791 

$502,444,353 

5.04% 

28 


Another  way  to  consider  this  same  data  is  represented  in  Figures  9  and  10. 
Figure  9  shows  the  total  number  of  ACC-APG  contract  actions  on  the  primary  Y-axis  and 
the  contract  actions  attributed  to  JPEO-CBD  on  the  secondary  Y-axis. 
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Figure  9.  ACC-APG  and  JPEO-CBD  Contract  Actions,  FY 12-FY15 


Figure  10  shows  the  total  ACC-APG  funding  obligations,  in  $M,  on  the  primary 
Y-axis  and  the  funding  obligations  attributed  to  JPEO-CBD  on  the  secondary  Y-axis,  also 
in  $M. 
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Figure  10.  ACC-APG  and  JPEO-CBD  Funding  Obligations,  FY12-FY15 
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Another  way  to  consider  this  same  data  is  by  the  individual  contracting  divisions. 
Although  there  are  nine  (9)  divisions  that  service  JPEO-CBD,  this  report  will  consider 
only  the  four  (4)  most  active  in  the  JPEO-CBD  portfolio.  These  divisions  are:  ACC- 
Belvoir,  ACC-Edgewood,  ACC-Natick,  and  ACC-APG  Tenant  The  remaining  five  (5) 
divisions  have  processed  fewer  than  five  (5)  actions  in  each  FY  for  the  JPEO-CBD. 

a.  ACC-Belvoir 

ACC-Belvoir,  located  at  Fort  Belvoir,  VA  processed  twenty-five  (25)  contracting 
actions  for  JPEO-CBD  between  FY12  and  FY15.  The  overall  JPEO-CBD  funding 
obligations  total  -$565,415,  meaning  that  the  contracting  office  largely  deobligated 
funding  from  contracts.  One  of  the  reasons  this  occurs  is  that  the  contract  has  reached  the 
end  of  its  period  of  performance,  and  the  contractor  did  not  spend  all  of  the  funds 
available.  This  happens  for  a  variety  of  reasons,  including  cost  under-runs  or  lessening 
the  scope  of  the  contract.  ACC-Belvoir  performed  zero  (0)  contract  actions  for  JPEO- 
CBD  in  FY  14.  This  data  is  presented  in  Figure  11. 


Figure  11.  ACC-Belvoir  Contract  Actions  and  Funding  Obligations  for  JPEO- 

CBD,  FY12-FY15 
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b.  ACC-Edgewood 

ACC-Edgewood,  located  at  Aberdeen  Proving  Ground,  MD  processed  1,020 
contracting  actions  for  JPEO-CBD  between  FY12  and  FY15.  The  overall  JPEO-CBD 
funding  obligations  total  $958.8M.  This  data  is  presented  in  Figure  12. 


Figure  12.  ACC-Edgewood  Contract  Actions  and  Funding  Obligations  for  JPEO- 

CBD,  FY12-FY 15 


c.  ACC-Natick 

ACC-Natick,  located  in  Natick,  MA,  processed  4,300  contracting  actions  for 
JPEO-CBD  between  FY12  and  FY15.  The  overall  JPEO-CBD  funding  obligations  total 
$1,264.6M.  This  data  is  presented  in  Figure  13. 
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Figure  13.  ACC-Natick  Contract  Actions  and  Funding  Obligations  for  JPEO- 

CBD,  FY12-FY15 


d.  ACC-APG  Tenant 

ACC-APG  Tenant,  also  located  at  Aberdeen  Proving  Ground,  MD,  processed 
sixty-five  (65)  contracting  actions  for  JPEO-CBD  between  FY12  and  FY15.  The  overall 
JPEO-CBD  funding  obligations  total  $5.0M.  This  data  is  presented  in  Figure  14. 
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Figure  14.  ACC-APG  Tenant  Contract  Actions  and  Funding  Obligations  for 

JPEO-CBD,  FY12-FY15 


e.  ACC-Combined 

For  a  more  comprehensive  look  at  the  four  (4)  divisions  that  service  JPEO-CBD 
most  often,  data  for  these  divisions,  hereafter  referred  to  as  APG-Combined,  is  combined 
and  presented  in  Figure  15. 


33 


1  800 

i  ouu 

1  ^00 

JLOUU 

1  ZLOO 

c*T 

•2  § 

1200 

U  m 

<  X 

1  000 

a  .2 

800 

•  JPEO  Actions 

a  w> 
o  s 

ouu 

600 

-•-JPEO  Obs,  $M 

Q  w 
«  ^ 
U  1 

^  r* 

400 

900 

o  2 

zuu 

n 

^9 

u 

FY12 

FY13 

FY14 

FY15 

Figure  15.  ACC-Combined  Contract  Actions  and  Funding  Obligations  for  JPEO- 

CBD,  FY12-FY15 

6.  JPEO-CBD  Contractual  Workload  Analysis 

Figures  9  through  15  show  a  variety  of  data  and  trends.  First,  compare  the  total 
number  of  contract  actions  of  ACC-APG  versus  the  number  of  contract  actions 
completed  for  JPEO-CBD  (Figure  9).  The  number  of  contract  actions  performed  by 
ACC-APG  appears  to  have  steadied  over  FY13-FY15,  while  the  number  of  JPEO-CBD 
contract  actions  has  continued  to  increase. 

Without  insight  into  the  full  scope  of  programs  ACC-APG  supports,  it  is  difficult 
to  determine  why  the  number  of  contract  actions  at  ACC-APG  has  leveled  off.  One 
possibility  is  that  there  is  less  work  being  performed  by  contractors  as  funding  resources 
have  become  scarcer,  meaning  that  fewer  contract  actions  are  required  to  initiate  or 
continue  work.  Another  possibility  is  that  various  program  offices  have  become  more 
efficient  in  their  contracting  approach,  and  thus  require  fewer  actions  to  complete  the 
planned  effort. 

The  growth  in  number  of  JPEO-CBD  contract  actions  shown  in  Figure  9  is 
clearer.  In  the  first  quarter  of  FY13,  JPEO-CBD  issued  its  first  omnibus  contract.  The 
JPEO-CBD  designed  its  Omnibus  Program,  Engineering  and  Technical  Support  services 
(OPETS)  mechanism  to  meet  the  intent  of  BBPs  1.0  and  2.0  by  reducing  overhead  costs 


34 


across  the  portfolio  and  increasing  competition  and  flexibility.  Task  orders  are  awarded 
as  Firm  Fixed  Price  awards,  which  reduces  the  risk  to  the  government. 

Through  ACC-Natick,  JPEO-CBD  issued  seventy  (70)  contract  awards  through 
OPETS.  Figure  13  shows  the  30%  growth  in  number  of  contract  awards  between  FY12 
and  FY13  at  ACC-Natick,  due  largely  to  the  initial  OPETS  award.  However,  the  total 
amount  of  funding  awarded  by  ACC-Natick  in  the  same  time  period  has  fallen.  It  is 
difficult  to  draw  conclusions  on  the  cause  of  the  decline  in  obligations,  and  whether  or 
not  it  is  a  consequence  of  the  BBP  initiatives  and  acquisition  reform  efforts.  This  decline 
in  obligations  may  also  be  due  to  declining  JPEO-CBD  total  obligation  authority  (TOA) 
in  those  years. 

Figure  10,  showing  ACC-APG  and  JPEO-CBD  funding  obligations,  provides  a 
similar  comparison.  Between  FY12  and  FY15,  total  funding  obligated  by  ACC-APG 
shows  a  clear  downward  trend  beginning  the  year  that  BBP  2.0  was  released,  although  it 
is  difficult  to  attribute  this  trend  to  the  measures  that  BBP  2.0  focused  on.  As  with  the 
ACC-Natick  data,  possible  explanation  could  be  that  obligations  decreased  due  to 
declining  TOA.  Further  analysis  is  needed  to  determine  if  there  is  a  correlation  between 
the  reforms  implemented  with  BBP  2.0  and  the  downward  obligation  trend  at  ACC-APG. 
There  is  also  no  apparent  reason  for  the  increase  in  JPEO-CBD  obligations  in  FY15,  as 
the  FY 15  TOA  did  not  increase  over  prior  years. 

Taken  individually,  Figures  11-14  do  not  show  any  clear  trends.  Combined  in 
Figure  15,  however,  the  data  becomes  clearer.  Between  FY12  and  FY15,  the  number  of 
contract  actions  attributed  to  JPEO-CBD  increased,  while  the  overall  funding  obligations 
decreased  over  the  same  time  period. 

One  of  the  likely  causes  of  this  trend  is  the  BBP  1.0  requirement  to  develop  the 
cost  estimate  showing  what  the  program  “should-cost.”  Should-cost  estimates  target 
controlling  program  cost  growth.  Within  the  CBD  enterprise,  the  new  should-cost 
requirement  presented  the  JPEO  and  JPMs  with  additional  challenges;  both  the  JPMs  and 
the  contractors  now  need  to  develop  two  (2)  cost  estimates.  This  manifests  in  the  data 
above  as  ACC-Combined  contracting  officers  had  to  issue  additional  contract 
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modifications  regarding  the  requirement  without  an  increase  in  funding  obligations.  This 
demonstrates  that,  at  least  in  the  short  term,  BBP  1.0  did  not  improve  efficiency  in  the 
acquisition  process  through  the  use  of  should-cost  management.  Instead,  workload  to 
execute  the  program  increased. 

Although  these  comparisons  indicate  that  BBP  1.0  rather  than  the  2015  DoDI 
5000.02  caused  a  downturn  in  program  efficiency,  it  is  essential  to  note  that  one  of  the 
primary  objectives  of  the  DoDI  5000.02  is  to  codify  policy  and  guidance  changes  made 
after  the  2008  DoDI  5000.02.  This  indicates  that,  at  a  minimum,  the  2015  DoDI  5000.02 
will  not  have  a  positive  impact  on  program  efficiency. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


As  fiscal  constraints  continue  to  impinge  upon  the  DOD’s  ability  to  field 
capability  to  the  warfighter,  changing  the  way  the  DOD  does  business  becomes 
increasingly  important.  The  2015  DoDI  5000.02  is  the  most  recent  acquisition  reform 
effort  to  increase  acquisition  process  effectiveness  and  efficiency. 

Data  from  the  2013  Annual  Report,  provided  in  Chapter  III.A.l,  shows  that  DOD 
contracts  have  been  trending  toward  decreasing  cost  for  over  twenty  (20)  years — well 
before  2015.  This  demonstrates  that  the  2015  DoDI  5000.02  is  likely  to  have  limited 
effect  on  driving  contract  costs  down  throughout  the  DOD,  as  they  began  declining  long 
before  the  release  of  the  updated  DoDI  5000.02. 

Although  more  work  and  understanding  are  needed  to  continue  to  reduce  the 
number  of  critical  and  significant  Nunn-McCurdy  breaches  described  in  Chapter  III.B.l, 
the  Department  of  Defense  has  implemented  stronger  guidance  and  policies  through  the 
release  of  DoDI  5000.02.  Specifically,  the  DOD  2015  report  states  that  “the  DOD  has 
made  significant  progress  in  improving  our  systems  engineering  practices.”  There  is  not 
yet  enough  data  available  to  support  the  assertion  that  significant  progress  has  been  made, 
although  it  may  in  time,  if  metrics  can  be  established  to  verify  that  effects  can  be 
demonstrated. 

Within  the  JPEO-CBD,  it  is  difficult  to  determine  the  likely  effect  on  contract 
cost.  Data  presented  in  Chapter  III.C.l  provides  limited  insight  into  how  DoDI  5000.02 
may  have  influenced  the  representative  program  schedules.  While  the  increased 
schedules  may  have  been  due  to  the  2015  document,  there  are  also  other  factors  in  play. 

A  more  clear  indication  that  DoDI  5000.02  may  not  have  the  desired  effect  of 
greater  efficiency  is  described  in  Chapter  III.C.3.  There  has  been  a  20%  increase  in  the 
number  of  documents  or  document  updates  required  at  various  milestones  and  decision 
points.  This  is  not  likely  to  increase  efficiency  within  the  acquisition  process.  More 
documents  require  more  time  and/or  more  resources  to  prepare  and  staff  the  documents. 
The  author  does  concede  that  some  of  the  changes  will  likely  increase  the  effectiveness  of 
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a  program  and  facilitate  positive  outcomes,  such  as  adding  OMS/MP  documentation 
requirement.  Overall,  however,  additional  documentation  requirements  will  likely  cause 
negative  schedule  impacts. 

The  data  presented  in  Chapter  III.C.5  demonstrate  that  CBD  enterprise  contract 
costs  have  been  decreasing  over  prior  years.  Between  FY12  and  FY15,  the  number  of 
contract  actions  attributed  to  JPEO-CBD  increased,  while  the  overall  funding  obligations 
decreased  over  the  same  time  period.  In  other  words,  increased  contract  workload 
obligated  fewer  funds.  Further  study  will  be  needed  in  future  years  to  determine  what 
impact  DoDI  5000.02  will  have  on  these  trends. 

Within  the  JPEO-CBD  community,  this  latest  effort  in  acquisition  reform  is  likely 
to  produce  few  positive  changes  for  program  efficiency  or  effectiveness.  While  the 
instruction  was  developed  to  “increase  emphasis  on  process  intent  and  thoughtful 
program  planning  rather  than  on  rules  and  compliance,”  this  is  subjective  (NDIA,  2015, 
p.  2).  The  JPEO-CBD  has  begun  placing  more  emphasis  on  tailoring  each  program,  yet 
program  tailoring  had  also  been  discussed  in  previous  versions.  Additionally,  tailoring 
often  involves  requesting  waivers  for  certain  requirements,  which  may  not  serve  to  drive 
efficiency  into  the  program  as  waiver  processes  can  be  equally  time-consuming. 
Moreover,  one  needs  to  determine  what  metrics  may  be  suitable  for  measuring  process 
intent  and  thoughtful  program  planning  to  determine  whether  the  2015  DoDI  5000.02  has 
been  successful. 

Additionally,  as  the  data  and  analysis  indicate,  there  has  been  an  increase  in  the 
work  required  to  bring  programs  through  development  and  into  fielding.  While  poor 
management  performance  remains  the  leading  cause  of  Nunn-McCurdy  program 
breaches,  program  and  contracting  offices  are  required  to  use  additional  resources  to 
accomplish  less  work  as  the  DOD  struggles  to  do  more  without  more. 

Developing  new  acquisition  reform  efforts  every  year  has  not  delivered 
capabilities  to  the  warfighter  faster.  In  many  cases,  these  efforts  may  have  had  the 
opposite  effect.  The  DOD  would  benefit  from  allowing  additional  time  between  issuing 
new  guidance  to  allow  for  increased  analysis  regarding  whether  the  previous  reform 
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efforts  were  successful  in  their  intent.  Additional  study  time  would  allow  the  DOD  to 
develop  more  effective  reform  initiatives.  This  can  be  difficult  to  accomplish,  however, 
given  that  acquisition  occurs  in  a  constantly  changing  environment. 

One  limitation  of  this  project  is  that  trends  can  be  difficult  to  analyze,  as  most  of 
them  are  based  on  lagging  indicators  such  as  historical  cost  and  schedule  trends.  The 
DOD  needs  to  determine  realistic  and  feasible  leading  indicators  that  can  predict  outcome 
and  instill  proactive  effort  rather  than  the  reaction  to  emerging  problems  that  is  necessary 
when  using  lagging  indicators. 

Another  limitation  is  the  lack  of  discussion  on  tailoring  various  aspects  of  a 
program  from  inception  through  development  to  fielding.  This  project  focused  solely  on 
contracts  and  funding.  Other  areas  of  study  that  might  provide  additional  insight  are  areas 
such  as  test  and  evaluation,  logistics,  and  systems  engineering.  For  example,  one  of  the 
known  complications  of  JPEO-CBD  program  schedules  and  funding  are  the  test 
requirements  imposed  by  the  Army  test  community.  JPEO-CBD  programs  are  largely 
ACAT  III,  yet  are  tested  to  the  level  and  with  the  thoroughness  of  ACAT  I  programs. 
These  requirements  drive  both  the  schedule  and  the  resources  required,  and  should  be 
studied  in  further  detail  to  provide  greater  insight  into  how  the  DoDI  5000.02’ s  emphasis 
on  tailoring  might  help  deliver  capabilities  to  the  warfighter  more  rapidly. 

Finally,  measuring  the  effectiveness  of  acquisition  reform  is  difficult  due  to  a 
number  of  factors.  Each  organization  or  program  office  conducts  acquisition  differently; 
targeting  one  variable  for  change  and  measuring  the  success  of  that  change  across 
different  military  services  and  organizations  is  impossible.  This  is  especially  true  when 
data  is  not  collected  nor  maintained  in  a  standard  manner. 

Therefore,  one  recommendation  is  to  consider  creating  a  standard  method  for  data 
collection  from  all  programs,  regardless  of  size.  Although  the  Defense  Acquisition 
Management  Information  Retrieval  (DAMIR)  database  is  intended  to  be  a  repository  for 
acquisition  program  information,  it  is  used  only  for  MDAPs  and  Major  Automated 
Information  Systems  (MAIS)  programs.  Expanding  the  DAMIR  database  to  include  more 
programs  would  provide  a  more  robust  look  at  the  data  and  trends  of  all  acquisition 
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efforts  rather  than  the  large-scale  efforts  of  MDAPs  and  MAIS  programs.  This  additional 
data  collection  burden  placed  on  small  program  offices  requires  a  cost-benefit  analysis 
prior  to  policy  change  implementation. 

Additionally,  while  the  Performance  of  the  Defense  Acquisition  System  reports 
were  valuable  in  discussing  performance  trends,  specifically  in  contract  performance,  it  is 
difficult  to  understand  whether  these  analyses  compare  directly  to  smaller  programs.  As 
with  the  DAMIR  database,  these  reports  focus  only  on  MDAP  and  MAIS  programs.  One 
recommendation  is  to  consider  expanding  the  scope  of  these  reports  in  future  iterations  to 
include  the  smaller  acquisition  programs.  Although  adding  significant  burden  to 
workload  of  level  II  and  III  programs,  this  would  allow  the  DOD  acquisition  community 
to  better  understand  the  implications  and  outcomes  of  reform  measures  more  completely. 
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